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REAL PARTY IN INTEREST 

The real party in interest in this application is Hewlett-Packard 
Development Company, L.P. 

RELATED APPEALS AND INTERFERENCES 

Appellant is unaware of any other related appeals or interferences that 
may directly affect or be directly affected by or have a bearing on the Board's 
decision in this appeal. 

STATUS OF THE CLAIMS 

Claims 1-8, 14-29, 32-41, 44, and 45 stand rejected under 35 U.S.C. 
§103(a) as being unpatentable over U.S. Patent Application 2004/0172255 of 
Aoki et al (" 'Aoki") and U.S. Patent Application 2004/0201710 of Uchihashi et al 
("Uchihashi"). 

Claims 9-13, 30, 31, 42, and 43 stand rejected under 35 U.S.C. §103(a) as 
being unpatentable over Aoki and Uchihashi and U.S. Patent Application 
2002/0191071 of Ruietal. ("Rui"). 

The examiner has objected to the specification under 37 CFR 
§1.75(d)(l) and MPEP §608.01(o) with respect to claims 34-45 and has objected 
to the drawings under 37 CFR §1.84(p)(5). 

Appellant appeals the rejection of all of the pending claims 1-45. Claims 
1-45 as currently pending are set forth in the attached Appendix. 

STATUS OF AMENDMENTS 

Appellant is unaware of any amendments filed after the Final Office 
Action mailed 9-5-09 which finally rejected claims 1-45. 
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SUMMARY OF CLAIMED SUBTECT MATTER 

Independent claims 1, 22, and 34 are directed to enabling multiple 
communication interactions to form among multiple individuals that are 
physically present in different environments by using respective sets of 
sensing and rendering components in the environments to detect the 
communication interactions and then capturing and combining the media 
data for the sensing and rendering components in response to the activities of 
the individuals associated with the respective communication interactions. 
(See pages 1-2 of Appellant's specification). 

Independent claim 1 is a system for communication including a first set 
of sensing and rendering components arranged to cover physical movements 
of multiple individuals present in a first environment (See Figure 5, element 
250 and pages 15-16 of Appellant's specification) and a second set of sensing 
and rendering components arranged to cover physical movements of 
multiple individuals present in a second environment (See Figure 5, element 
252 and pages 15-16 of Appellant's specification). A system according to claim 
1 includes an interest thread detector (See Figure 1, element 16 and page 5 of 
Appellant's specification) that uses the first and second set of sensing and 
rendering components to detect multiple communication interactions each 
involving a respective subset of the individuals present in the first and second 
environments (See Figure 1, element 16 and page 5, last paragraph of 
Appellant's specification) and that maintains an interest thread for each 
communication interaction (See Figure 1, element 16 and page 5, last 
paragraph of Appellant's specification) and further includes a communication 
provider (See Figure 1, element 18 and page 6, first paragraph of Appellant 7 s 
specification) that captures a set of media data from the sensing components 
and that combines the captured media data in response to the respective 
activities of each subset of the individuals (See Figure 2, element 36 and page 
6, fourth paragraph of Appellant's specification) and that communicates the 
combined media data to the rendering components (See Figure 2, element 38 
and page 6, fourth paragraph of Appellant's specification). 

Independent claim 22 is directed to a method for communication that 
includes providing a first set of sensing and rendering components for 
covering physical movements of multiple individuals present in a first 
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environment (See Figure 5, element 250 and pages 15-16 of Appellant s 
specification) and providing a second set of sensing and rendering 
components for covering physical movements of multiple individuals present 
in a second environment (See Figure 5, element 252 and pages 15-16 of 
Appellant's specification). A method according to claim 22 further includes 
detecting multiple communication interactions each a communication 
interaction involving a respective subset of the individuals present in the first 
and second environments (See Figure 2, element 30 and page 6, second 
paragraph of Appellant's specification) and maintaining an interest thread for 
the each detected communication interaction (See Figure 2, element 32 and 
page 6, second paragraph of Appellant s specification) and capturing a set of 
media data from the sensing components (See Figure 2, element 34 and page 
6, fourth paragraph of Appellant's specification) and combining the captured 
media data in response to the respective activities of the respective subset of 
the individuals (See Figure 2, element 36 and page 6, fourth paragraph of 
Appellant's specification) and communicating the combined media data to the 
rendering components (See Figure 2, element 38 and page 6, fourth 
paragraph of Appellant 7 s specification). 

Independent claim 34 is directed to a computer-readable storage media 
that includes providing a first set of sensing and rendering components for 
covering physical movements of multiple individuals present in a first 
environment (See Figure 5, element 250 and pages 15-16 of Appellant 7 s 
specification) and providing a second set of sensing and rendering 
components for covering physical movements of multiple individuals present 
in a second environment (See Figure 5, element 252 and pages 15-16 of 
Appellant's specification). A method according to claim 22 further includes 
detecting multiple communication interactions each a communication 
interaction involving a respective subset of the individuals present in the first 
and second environments (See Figure 2, element 30 and page 6, second 
paragraph of Appellant's specification) and maintaining an interest thread for 
the each detected communication interaction (See Figure 2, element 32 and 
page 6, second paragraph of Appellant's specification) and capturing a set of 
media data from the sensing components (See Figure 2, element 34 and page 
6, fourth paragraph of Appellant's specification) and combining the captured 
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media data in response to the respective activities of the respective subset of 
the individuals (See Figure 2, element 36 and page 6, fourth paragraph of 
Appellant's specification) and communicating the combined media data to the 
rendering components (See Figure 2, element 38 and page 6, fourth 
paragraph of Appellant's specification). 
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GROUNDS OF REIECTTON TO BE REVIEWED ON APPEAL 

I: Rejection of claims 1-8, 14-29, 32-41, 44, and 45 as being 
obvious in view of Aoki and Uchihashi. 

II: Rejection of claims 9-13, 30, 31, 42, and 43 as being 
obvious in view of Aoki and Uchihashi and Rui. 

Ill: Objection to the specification under 37 CFR §1. 75(d)(1) 
and MPEP §608.01 (o). 

IV: Objection to the drawings under 37 CFR §1.84(p)(5). 
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ARGUMENT 



I: Claims 1-8, 14-29, 32-41, 44, and 45 are not obvious in 
view of Aoki and Uchihashi because Aoki and Uchihashi do 
not disclose or suggest the limitations of independent 
claims 1, 22, and 34. 

Appellant respectfully submits that independent claims 1, 22 and 34, 
and claims 2-8, 14-21, 23-29, 32-33, 35-41, and 44-45 which depend from 
independent claims 1, 22 and 34, are not obvious in view of Aoki and Uchihashi 
because Aoki and Uchihashi do not disclose or suggest detecting multiple 
communication interactions involving the physical movements of multiple 
individuals in respective first and second environments and then combining 
media data in response to activities indicated by their physical movements as 
claimed in claims 1, 22, and 34. Instead, Aoki teaches detecting different 
conversations, called conversational floors 1 , carried on a shared 
communication channel 2 such as a telephone party line {Aoki, paragraphs 
0006-0007) by analyzing the timing of vocalizations carried on the shared 
communication channel {Aoki, paragraph 0035) and Uchihashi discloses video 
camera tracking of individuals speaking in a meeting room {Uchihashi, 
paragraph 0008). 

The examiner has stretched the limit of reason beyond its breaking 
point in applying the conversational floors of Aoki to the physical 
environments of claims 1, 22, and 34 by stating that Aoki teaches "detecting 
conversational characteristics... of multiple individuals present 3 in a (first 
environment) conversational floor. . (Page 5, Office Action, mailed 9-5-08) 
(emphasis added). It is respectfully submitted that a conversational floor as 
taught by Aoki is not an environment as claimed in claims 1, 22, and 34 
because a conversational floor as taught by Aoki is a sub-conversation carried 
on a shared communication channel {Aoki, paragraph 0006) whereas an 
environment as claimed in claims 1, 22, and 34 is a physical environment 

1 Appellant will show that the examiner has imparted a non-existent physicality into the 
term "conversational floor" of Aoki. 

2 Aoki discloses a telephone party line, instant messaging, and a computer chat room as 
examples of a shared communication channel. (Aoki, paragraphs 0007-0008) 

3 It is impossible for an individual to be present in a conversational floor as stated by the 
examiner because a conversational floor as disclosed in Aoki is not a physical environment 
like an environment as claimed in claims 1, 22, and 34. 
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having sensing and rendering components that cover the physical 
movements and activities of multiple individuals present in the environment. 
It is therefore submitted that individuals cannot be "present" in a 
conversational floor of Aoki in the way that individuals are physically present 
in an environment as claimed in claims 1, 22, and 34. 

The examiner has acknowledged that Aoki does not teach detecting 
physical movements of individuals but has stated that Uchihashi does and that 
it would have been obvious to incorporate the physical movement detection 
of Uchihashi into the system of Aoki. (Page 8, Office Action, mailed 9-5-08). It is 
submitted that any such combination of would lack the limitation of detecting 
multiple communication interactions involving the physical movements of 
multiple individuals in respective first and second environments and then 
combining media data in response to activities indicated by their physical 
movements as claims in claims 1, 22, and 34. Uchihashi does not teach 
detecting multiple communication interactions involving the physical 
movements of multiple individuals in respective first and second 
environments and then combining media data in response to activities 
indicated by their physical movements as claimed in claims 1, 22, and 34. 
Instead, the source analyzer controller 30 of Uchihashi only assists an operator 
in pointing a camera toward an activity of interest in a room. ("The candidate 
event activity is then provided to the operator in an intuitive format 
facilitating the selection of the appropriate camera capable of capturing the 
second speaker." Uchihashi at paragraph 0022). 

It is submitted that incorporation of the video tracking taught by 
Uchihashi into the system of Aoki for parsing conversations carried on a 
shared communication channel would yield a system that uses video tracking 
of individuals to parse conversations underway on a shared communication 
channel rather a system for detecting and tracking multiple communication 
interactions involving multiple individuals present in multiple physical 
environments as claimed in claims 1, 22, and 34. This conclusion is supported 
by the teaching in Aoki that the locations of users if known may be used to 
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parse the shared communication channel 4 conversations. (Aoki, paragraph 
0168). 

The examiner has stated that the candidate event activities of Uchihashi 
anticipate an interest thread as claim in claims 1, 22, and 34. (Page 8, Office 
Action, mailed 9-5-08). It is respectfully submitted that a display of candidate 
event activity to a camera operator as taught by Uchihashi does not anticipate 
an interest thread for a communication interaction involving a respective 
subset of the individuals present in first and second environments as claim in 
claims 1, 22, and 34. Instead, the candidate event activities of Uchihashi are 
used to suggest camera movements to a camera operator. (Uchihashi, 
paragraph 0022). 



4 Aoki in paragraph 0168 states that users present in the same building are more likely to be 
involved in a conversation with each other than users in remote locations who "are present 
in the computer-mediated communication system but not in the room/' It is submitted that 
the term "computer-mediated communication system" is a shared communication channel. 
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II: Claims 9-13, 30, 31, 42, and 43 are not obvious in view of 
Aoki and Uchihashi and Rui because Aoki and Uchihashi 
and Rui do not disclose or suggest the limitations of 
independent claims 1, 22, and 34. 

Appellant respectfully submits that claims 9-13, 30, 31, 42, and 43 are 
not obvious in view of Aoki and Uchihashi and Rui because Aoki and Uchihashi 
and Rui do not disclose or suggest detecting multiple communication 
interactions involving the physical movements of multiple individuals in 
respective first and second environments and then combining media data in 
response to activities indicated by their physical movements as claimed in 
claims 1, 22, and 34 from which claims 9-13, 30, 31, 42, and 43 depend. 
Appellant has shown that Aoki and Uchihashi do not teach or suggest detecting 
multiple communication interactions involving the physical movements of 
multiple individuals in respective first and second environments and then 
combining media data in response to activities indicated by their physical 
movements as claimed in claims 1, 22, and 34. Rui teaches aiming a camera in 
response to user selections (see abstract and figure 1 of Rui) rather than 
detecting multiple communication interactions involving the physical 
movements of multiple individuals in respective first and second 
environments and then combining media data in response to activities 
indicated by their physical movements as claimed in claims 1, 22, and 34. 
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Ill: Appellant's specification satisfies 37 CFR §1 -75(d)(1) and 
MPEP §608.01(o) with respect to the term "computer 
readable storage media" recited in claims 34-45. 

The examiner has stated that 

The specification is objected to as failing to provide proper 
antecedent basis for the claimed subject matter. See 37 CFR §1 .75(d)(1) 
and MPEP §608.01(o). 

Correction of the following is required: Applicant claims a 
" computer-readable storage media" in claims 34-45, which is not 
defined in the specification. 
(Page 3, Office Action, mailed 9-5-08) (emphasis added). 

It is respectfully submitted that 37 CFR §1.75(d)(l) and MPEP 
§608.01 (o) do not require that the individual term "computer-readable 
storage media" have an antecedent basis in the specification or be defined in 
the specification as stated by the examiner. Instead, 37 CFR §1. 75(d)(1) states 
that the claim terms must find clear "support or antecedent basis" (emphasis 
added) and MPEP §608.01(o) states that individual terms should "have clear 
support or antecedent basis in the specification" (emphasis added). MPEP 
§2111.01 provides that claim terms are to be given their plain meaning to one 
of ordinary skill in the art. 5 

It is submitted that appellant's specification provides clear support for a 
computer readable storage media as claimed in claims 34-45 because any 
person skilled in the pertinent art would be well-knowledgeable of how to 
make and use a computer readable storage media that contains a set of code 
that when executed provides communication according to the steps recited in 
claims 34-45 from reading appellanf s specification. Appellant 7 s specification 
provides ample descriptions of how to detect multiple communication 
interactions and how to capture and combine media data from the sensing 
components as claimed in claims 34-45. Appellant's invention lies in the steps 
carried out by the code stored on a computer-readable storage media and not 
in the nature of the computer-readable storage media itself. 6 



5 MPEP §2111.01 provides that claim terms are to be given their plain meaning to one of 
ordinary skill in the art unless such plain meaning is contradicted by the specification and 
nothing in appellant's specification contradicts the plain meaning of "computer readable 
storage media/ 7 

6 Appellant is at a loss to understand what interest or purpose, public or private, would be 
served by requiring appellant to define "computer-readable storage media" in appellant's 
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IV: Appellant's drawings satisfy 37 CFR $1.84(p)(5). 

The examiner has stated that 

The drawings are objected to as failing to comply with 37 CFR 
§1.84(p)(5) because they do not include the following reference signs(s) 
mentioned in the description: the description of figure 3 in the 
specification includes an element 13, which is not included in the 
drawing. 7 

(Page 3, Office Action, mailed 9-5-08) (emphasis original). 

37 CFR §1.84(p)(5) in pertinent part states that "Reference characters 
mentioned in the description must appear in the drawings/ 7 Accordingly, 
appellant's description mentions a rich media environment 13 which is shown 
in Figure 1 of appellant's drawings. 

It is submitted that the bold emphasis of "figure 3" used by the 
examiner is meant to convey that the examiner believes that the rich media 
environment 13 must appear in Figure 3 because it is mentioned after Figure 
3 is mentioned in the description. It is submitted that no such requirement is 
imposed by 37 CFR §1.84(p)(5) or by any other rule. 8 



specification given the plain meaning of the term to one of ordinary skill in the art and 
given the subject matter of appellants invention. 

7 It is submitted that by "the drawing" the examiner refers to Figure 3 and so the examiner 
is demanding that although element 13 is shown in Figure 1 it must also be shown in Figure 
3. 

8 Likewise, appellant is at a loss to understand what interest or purpose, public or private, 
would be served by requiring appellant to include the rich media environment 13 in Figure 3 
given that it is shown in Figure 1 and is well-defined in the description. 
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CONCLUSION 

Appellant respectfully submits that the stated rejections and objections 
cannot be maintained in view of the arguments set forth above. Appellant 
respectfully submits that all of the claims 1-45 are patentable under 35 U.S.C. 
§103 over the references cited by the Examiner and requests that the Board of 
Patent Appeals and Interferences direct allowance of the rejected claims. 

Respectfully submitted, 
By 

Date: ^-S^ f W (\Az^ 

Paul ft. Horsimann " 
Reg. No. 36,167 
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CLAIMS APPENDIX 

1. A system for communication, comprising: 

first set of sensing and rendering components arranged to cover 
physical movements of multiple individuals present in a first environment; 

second set of sensing and rendering components arranged to cover 
physical movements of multiple individuals present in a second environment; 

interest thread detector that uses the first and second set of sensing 
and rendering components to detect multiple communication interactions 
each involving a respective subset of the individuals present in the first and 
second environments and that maintains an interest thread for each 
communication interaction; 

communication provider that captures a set of media data from the 
sensing components and that combines the captured media data in response 
to the respective activities indicated by physical movements of each subset of 
the individuals and that communicates the combined media data to the 
rendering components. 

2. The system of claim 1, wherein the communication provider selects a 
respective subset of the first and second set of sensing and rendering 
components for use for each interest thread. 

3. The system of claim 1, wherein the respective activities include speech 
levels of the individuals involved in the respective interest thread. 

4. The system of claim 1, wherein the respective activities include 
gestures by the individuals involved in the respective interest thread. 

5. The system of claim 1, wherein the respective activities include 
movements by the individuals involved in the respective interest thread. 

6. The system of claim 1, wherein the respective activities include 
locations of the individuals involved in the respective interest thread. 
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7. The system of claim 1, wherein the communication provider refines 
the media data obtained from the sensor components in response to the 
respective activities. 

8. The system of claim 1, wherein the communication provider stores the 
combined media data to provide a history of each communication interaction. 

9. The system of claim 1, wherein the communication interactions include 
a communication interaction that pertains to an artifact in one of the 
environments. 

10. The system of claim 9, wherein the artifact changes over time. 

11. The system of claim 9, wherein the artifact is a shared virtual writing 
surface. 

12. The system of claim 10, wherein a change to the artifact is made by one 
of the individuals involved in the interest threads. 

13. The system of claim 10, wherein the communication provider records a 
history of the artifact over time. 

14. The system of claim 1, wherein the interest thread detector detects one 
or more activities in the environments and creates an interest area for the 
detected activity. 

15. The system of claim 14, wherein the interest thread detector associates 
the interest area with another interest thread. 

16. The system of claim 1, wherein the communication interactions include 
a communication interaction that involves two or more of the individuals in 
one of the environments. 



Appeal Brief 

Application Ser. No.: 10/682,542 



16 



Attorney Docket No.: 200315123-1 



17. The system of claim 1, wherein the communication interactions include 
a communication interaction that involves one or more of the individuals in 
two of the environments. 

18. The system of claim 1, wherein the interest thread detector detects 
formation by detecting a movement of one of the individuals. 

19. The system of claim 18, wherein the movement pertains to one of the 
rendering devices. 

20. The system of claim 18, wherein the movement pertains to one of the 
other individuals. 

21. The system of claim 1, wherein one or more of the individuals is in a 
remote location and in possession of a remote sensing and rendering 
component. 

22. A method for communication, comprising: 

providing a first set of sensing and rendering components for covering 
physical movements of multiple individuals present in a first environment; 

providing a second set of sensing and rendering components for 
covering physical movements of multiple individuals present in a second 
environment; 

detecting multiple communication interactions each a communication 
interaction involving a respective subset of the individuals present in the first 
and second environments; 

maintaining an interest thread for the each detected communication 
interaction; 

capturing a set of media data from the sensing components; 
combining the captured media data in response to the respective activities 
indicated by physical movements of the respective subset of the individuals; 
communicating the combined media data to the rendering components. 



Appeal Brief 

Application Ser. No.: 10/682,542 



17 



Attorney Docket No.: 200315123-1 



23. The method of claim 22, further comprising selecting a respective 
subset of the sensing and rendering components for use for each interest 
thread. 

24. The method of claim 22, wherein combining the captured media data 
includes detecting speech levels of the corresponding individuals. 

25. The method of claim 22, wherein combining the captured media data 
includes detecting gestures by the corresponding individuals. 

26. The method of claim 22, wherein combining the captured media data 
includes detecting movements by the corresponding individuals. 

27. The method of claim 22, wherein combining the captured media data 
includes detecting locations of the corresponding individuals. 

28. The method of claim 22, further comprising refining the media data 
obtained from the sensor components in response to the respective activities. 

29. The method of claim 22, further comprising storing the combined 
media data in a history of each communication interaction. 

30. The method of claim 22, further comprising monitoring an artifact 
over time. 

31. The method of claim 30, further comprising recording a history of the 
artifact over time. 

32. The method of claim 22, further comprising detecting one or more 
activities in the environments and creating an interest area for each detected 
activity. 

33. The method of claim 32, further comprising associating the interest 
area with another interest thread. 
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34. A computer-readable storage media that contains a set of code that 
when executed provides communication by: 

providing a first set of sensing and rendering components for covering 
physical movements of multiple individuals present in a first environment- 
providing a second set of sensing and rendering components for 
covering physical movements of multiple individuals present in a second 
environment; 

detecting multiple communication interactions each involving a 
respective subset of the individuals present in the first and second 
environments; 

maintaining an interest thread for each detected communication interaction; 

capturing a set of media data from the sensing components; 
combining the captured media data in response to the respective activities 
indicated by physical movements of the respective subset of the individuals; 
communicating the combined media data to the rendering components. 

35. The computer-readable storage media of claim 34, further comprising 
selecting a respective subset of the sensing and rendering components for use 
for each interest thread. 

36. The computer-readable storage media of claim 34, wherein combining 
the captured media data includes detecting speech levels of the corresponding 
individuals. 

37. The computer-readable storage media of claim 34, wherein combining 
the captured media data includes detecting gestures by the corresponding 
individuals. 

38. The computer-readable storage media of claim 34, wherein combining 
the captured media data includes detecting movements by the corresponding 
individuals. 



Appeal Brief 

Application Ser. No.: 10/682,542 



19 



Attorney Docket No.: 200315123-1 



39. The computer-readable storage media of claim 34, wherein combining 
the captured media data includes detecting locations of the corresponding 
individuals. 

40. The computer-readable storage media of claim 34, further comprising 
refining the media data obtained from the sensor components in response to 
the respective activities. 

41. The computer-readable storage media of claim 34, further comprising 
storing the combined media data in a history of each communication 
interaction. 

42. The computer-readable storage media of claim 34, further comprising 
monitoring an artifact over time. 

43. The computer-readable storage media of claim 42, further comprising 
recording a history of the artifact over time. 

44. The computer-readable storage media of claim 34, further comprising 
detecting one or more activities in the environments and creating an interest 
area for each detected activity. 

45. The computer-readable storage media of claim 44, further comprising 
associating the interest area with another interest thread. 



Appeal Brief 

Application Ser. No.: 10/682,542 



20 



Attorney Docket No.: 200315123-1 



EVIDENCE APPENDIX 

None. 
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RELATED PROCEEDINGS APPENDIX 

None. 
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